Systems and methods relating to AFIS recognition, extraction, and 3-D analysis strategies

ABSTRACT

Systems, methods, etc., that assist a print examiner to thoroughly search and compare a print or substantial portion thereof against a known print database contained within an AFIS system. In certain embodiments, the prints can be definitively matched to a corresponding same print in the database. A result of a more thorough search and comparison can be a higher hit score and accuracy rate. In certain embodiments, the database comprises a candidate list of previously obtained prints to assist in the identification.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority from U.S. provisional patentapplication No. 60/513,669, filed Oct. 23, 2003; U.S. provisional patentapplication No. 60/517,849, filed Nov. 6, 2003; U.S. provisional patentapplication No. 60/518,263, filed Nov. 7, 2003; U.S. provisional patentapplication No. 60/548,214, filed Feb. 27, 2004; U.S. provisional patentapplication No. 60/562,635, filed Apr. 14, 2004; U.S. provisional patentapplication No. 60/572,665, filed May 19, 2004; U.S. provisional patentapplication No. 60/582,414, filed Jun. 23, 2004; and, U.S. provisionalpatent application No. 60/604,092 filed Aug. 23, 2004; which areincorporated herein by reference in their entirety and for all theirteachings and disclosures.

BACKGROUND

Automated Fingerprint Identification Systems (AFIS) systems,http://onin.com/fp/afis/afis.html, have not traditionally been able tosearch and match all fingerprints that human examiners can identifythrough manual methods. In part this is because human latent printexaminers can use additional detail, such as level III detail, to matchfingerprints (as used herein, “print” refers to unique identifyingprints on an animal, typically a human, such as fingerprints, palmprints, toe prints, foot prints, etc.). At the same time, the physicallimitations of the human eye prevent human examiners from distinguishingbetween very fine levels of grayscale magnitudes, which interferes bothwith identification of locations for, and the accuracy of, minutiaemarkers in each of level I, II and III details in fingerprints. Level 3detail is the finest-detail of the three levels. Generally speaking,level 1 is directed to ridge flow in prints, level 2 to ridge path inprints, and level 3 to ridge shape, which comprises unique edge detail,pore shape and position, incipient ridge shape, and other detail smallerthan a ridge width. Thus, existing systems do not always allow AFISsystems, operators and examiners to account for, and mark as minutiapoints, features that can be accurately discerned by visualizing subtledifferences in grayscale magnitude in a fingerprint image.

AFIS systems have not been developed to recognize, extract, quantify,search, and match level III characteristics in two or more fingerprintimages. AFIS systems also have not developed the capacity to identifyand mark desired minutiae markers at level III detail. Without thiscapability, subtle grayscale magnitude differences cannot be thoroughlyanalyzed, and therefore certain features may not be recognized,extracted, quantified, searched, and/or matched. Moreover, AFIS IIIdetail may reside at different grayscale magnitudes (or other magnitudessuch as hue or saturation) in a fingerprint image. If grayscalemagnitudes are not considered, certain level III features may not berecognized, extracted, quantified, searched, and/or matched. (Note: AFISIII is often used as shorthand herein to indicate level III analysis ofprints whether actually performed on in an automated system ormanually.)

Thus, there has gone unmet a need for additional AFIS system tools toassist in distinguishing between features that reside at differentgrayscale magnitudes, and to mark those features in a manner that can besearched on the AFIS system. The present systems, methods, etc., providethese and other advantages.

SUMMARY

The present systems and methods are, in some embodiments, referred toherein as AFIS 3-D or AFIS III+ for various embodiments, and assist anexaminer to thoroughly search and compare a print or substantial portionthereof against a known print database contained within an AFIS system.The substantial portion comprises enough of the print to be able toidentify adequate features to search and compare with the print againsta known print database, and in some embodiments comprises enough of theprint to at least tentatively match the print with at least one of theknown prints. In certain embodiments, the prints can be definitivelymatched to a corresponding same print in the database. A result of amore thorough search and comparison can be a higher hit score andaccuracy rate. In certain embodiments, the database comprises acandidate list of previously obtained prints to assist in theidentification.

One significant benefit provided herein is more accurate comparison oflatent print to known print images (“print” includes prints of anyappropriate body part, such as lips, skin, fingerprints, palm prints,toe prints, etc.). For example, a bomb fragment may contain only onesmall partial latent print. The more accurate AFIS 3-D and/or III+systems can be more likely to result in an identification of the sourceof the latent print based on a search against a database ofprints/images.

The present innovation can help to identify and neutralize threats tonational homeland security, whether those threats are foreign ordomestic, for example by searching an image/print in a more accuratemanner using subtle grayscale and/or other magnitudes in that image tocreate more comprehensive and accurate AFIS markers such as minutiaemarkers (AFIS level 2 or 3 markers), and/or b y searching an image/printmultiple times using different magnitudes in that image to createdifferent pathways which are then used to recognize, extract, quantify,search, and match a comprehensive set of level III features.

Thus, in some embodiments the methods and related software and othersystems herein comprise analyzing prints comprising: a) providing an atleast 2-dimensional image of a print; b) subjecting the image tomagnitude enhancement analysis such that at least one relativemeasurement across at least a substantial portion of the print can bedepicted in an additional dimension relative to the at least2-dimensions to provide a magnitude enhanced image such that additionallevels of that magnitude can be substantially more cognizable to a humaneye compared to the 2-dimensional image without the magnitudeenhancement analysis; c) displaying the enhanced image; and d) manuallyreviewing the magnitude enhanced image to place at least one minutiamarker on the print.

The placing can comprise identifying and placing at least one, two ormore minutiae markers not previously identified on the print, and cancomprise moving at least one, two or more minutiae markers previously,incorrectly placed on the print.

The methods can also comprise analyzing prints, comprising: a) providingan at least 2-dimensional image of a print comprising minutiae markersdetermined by an automated minutia marker algorithm to provide automatedminutiae markers; b) subjecting the image to magnitude enhancementanalysis such that at least one relative magnitude across at least asubstantial portion of the print can be depicted in an additionaldimension relative to the at least 2-dimensions to provide an magnitudeenhanced image such that additional levels of the chosen measure ormagnitude can be visible to a human eye compared to the 2-dimensionalimage without the magnitude enhancement analysis; c) displaying theenhanced image; and d) manually reviewing the magnitude enhanced imageto evaluate the correctness of the automated minutiae markers. Themethods can further comprise placing at least one minutia marker of theprint, the placing comprising at least one of removing incorrectautomated minutiae markers, moving incorrect automated minutiae markers,or adding further minutiae markers.

In another aspect, the methods herein comprise analyzing printscomprising: a) providing an at least 2-dimensional image of a print; b)subjecting the image to magnitude enhancement analysis such that atleast one relative magnitude across at least a substantial portion ofthe print is depicted in an additional dimension relative to the atleast 2-dimensions to provide a magnitude enhanced image such thatadditional levels of magnitude are visible to a human eye compared tothe 2-dimensional image without the magnitude enhancement analysis; c)dividing the magnitude enhanced image into a plurality of intensitylevels; d) individually selecting at least one isolated intensity level;and e) determining at least one AFIS marker from the isolated intensitylevel. The methods can further comprise displaying the isolated leveland manually or automatically determining the at least one minutiamarker.

These and other aspects, features and embodiments are set forth withinthis application, including the following Detailed Description andfigures. Unless expressly stated otherwise or clear from the context,all embodiments, aspects, features, etc., can be mixed and matched,combined and permuted in any desired manner. In addition, variousreferences are set forth herein that discuss certain systems, apparatus,methods and other information; all such references are incorporatedherein by reference in their entirety and for all their teachings anddisclosures, regardless of where the references may appear in thisapplication.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a screen shot of an image of a print being analyzed accordingto methods herein. The image comprises 4 automatically extractedfeatures, only 1 is correct. The other 3 features (on the right) are notcorrect.

FIG. 2 depicts the same latent print as FIG. 1 rendered using softwarethat depicts image intensity characteristics as a 3D surface.

FIGS. 3–5 are a series of screen shots of a portion of the image inFIGS. 1 and 2 demonstrating dynamic views of 3-dimensional ridge shapewherein the z-axis indicates grayscale magnitudes.

FIG. 6 is a screen shot of a 2D palm print rendered normally and with anintensity magnitudes shown on an additional dimension.

FIG. 7 is a screenshot of a fingerprint plotted with an embodiment ofthe 3D minutia marking tools discussed herein.

FIG. 8 depicts the same palm print as FIG. 6, going through the minutiamarking process.

FIG. 9 depicts the same palm print as FIG. 6 undergoing roll, tilt andpan.

FIG. 10 depicts the same palm print image as FIG. 6 viewed throughenhancement filters such as contrast adjustments further increase visualclarity.

FIG. 11 depicts the same palm print image as FIG. 6 wherein a user hasplaced minutiae markers on the 3D surface that can be quantified andsearched against existing 2D databases.

FIG. 12 a depicts an image of a known print impression.

FIG. 12 b depicts an AFIS II system reading ridge paths and based onmajor ridge path deviations.

FIG. 12 c depicts an AFIS II system recognizing and extracting level IIdetail with directionality and relationship.

FIGS. 13 a–c depict the same images as FIGS. 12 a–c with AFIS level IIIdetail demonstrated.

FIG. 14 depicts an AFIS III+ topographical analysis of a portion of thesame print as in the images in FIGS. 12 a–c at different grayscalemagnitudes.

FIGS. 15 a–c depict an AFIS III+ analysis the same images as FIGS. 12a–c at different grayscale magnitudes.

FIGS. 16 a–c depict an AFIS III+ analysis the same images as FIGS. 12a–c at a different grayscale magnitude from the analysis in FIGS. 15a–c.

FIG. 17 is a table demonstrating the progressive nature of thecombination of slices from multiple levels and multiple images.

FIG. 18 depicts images comprising examples of a concave morphologicalfeature and a convex morphological feature in an image of a print.

FIG. 19 depicts images comprising examples of a edge features in animage of a print.

DETAILED DESCRIPTION

AFIS 3-D Visualization Methods and Systems

When the full range of grayscale values in a fingerprint image arethoroughly analyzed by a human examiner, the placement of markers on theimage that occurs as part of an AFIS system search will be moreaccurate. A series of markers that have been placed more accurately,and/or in greater number, than would be possible absent analysis of thefull range of grayscale or other magnitudes, may increase the chances ofan AFIS hit.

Two problems with current AFIS analysis modules (which do not permitanalysts to place minutiae markers based on grayscale magnitudesvisualized on a 3-D surface) is that all relevant features present in animpression may not be recognized by the examiner, or the exact positionof some features may not be correctly determined. Software and systemsthat overcome the human visual system's weakness at discerning imagegrayscale magnitudes, by portraying those values as a 3D surface,assists a fingerprint examiner to discern very subtle variations inimage grayscale magnitudes and place AFIS markers in the correctlocation and position.

AFIS systems generally recognize detail in two ways: 1) automatic,system-generated placement of fingerprint minutiae markers, and 2)placement of minutiae markers by human analysts. In addition, the humanexaminer can place additional markers on a print image to supplementmarkers automatically generated by the AFIS system, and vice-versa, andthe human examiner can move or delete improper markers on the printimage to correct improper markers generated automatically by the AFISsystem. AFIS 3-D and AFIS III+ assist in the human detection andplacement of detail and minutia markings in a fingerprint image or otherprint image, thereby resulting in improved results in the AFIS search.

In FIG. 1, a latent print 30 was processed through the automaticextraction and minutia placement feature of a Universal LatentWorkstation (ULW). This software is used by law enforcement agenciesnationwide to submit latent print searches to the FBI IAFIS system. Thethree minutiae markers 32 on the right side of the impression do notaccurately reflect ridge structure underneath the marking. They areincorrect. The minutia marker 33 on the left is correct By visualizingthis impression as a 3-D surface, preferably dynamically (i.e., wherethe 3-D surface can be rolled, tilted and panned, also known as pitched,yawed and/or rolled, and can even be incorporated into a cine loop suchthat the 3-D surface is pitched, yawed and/or rolled in a reiterativemanner), the ridge path in this area can be followed through this areawithout splitting or coming to an end.

In FIG. 2, the same latent print was rendered in software that depictsimage intensity characteristics as a 3D surface. In particular, thedigital image of FIG. 1 was provided for rendering, then the image wassubjected to intensity value enhancement analysis such that at least onerelative intensity value across at least a substantial portion of theprint was depicted in an additional dimension relative to the at least2-dimensions. This provided an intensity value enhanced image such thatadditional levels of intensity values were substantially more cognizableto a human eye compared to the 2-dimensional image without the intensityvalue enhancement analysis. Cognizable indicates that in someembodiments the feature was not visible without the enhancement, but inother embodiments the feature was visible without the enhancement butwas substantially more cognizable, or more quickly cognizable, as afeature with the enhancement. Through a detailed examination of thisnature, many more correct characteristics were visualized, andadditional AFIS minutiae markers 34 were plotted. In FIG. 2, forexample, 15 characteristics were visualized on the dynamic 3D surfacemap and plotted with minutiae markers 34 on the image.

FIGS. 3–5 are a series of screen shots of a portion 36 of the image inFIGS. 1 and 2 demonstrating dynamic (pan, tilt or roll) AFIS 3-Danalyses of a 3-dimensional ridge shape wherein the z-axis indicatesgrayscale magnitudes, giving the examiner more information to visualizeand therefore creating an opportunity to place more minutiae markers,and do so with more accuracy. In certain embodiments in this and otheraspects of the invention, the image or desired portions thereof can alsobe enlarged (zoomed), reduced, etc.

The additional, correctly placed minutiae markers depicted in FIG. 2,when saved into the file format used by an AFIS system, are thensearched against the system's fingerprint database, using an AFISsystem's matching algorithms. The AFIS system then generates a report onthe probability of a match, and if so matched, assigns a relative scoreon the strength of that probability. As a general rule, the higher thenumber of correct minutiae markers, (i) the greater the probability thatan AFIS “hit” will occur, and (ii) the higher the “confidence level”score that is issued by an AFIS system for each hit.

The AFIS extra-dimension systems, devices, methods, etc., herein couldbe used in a variety of configurations, including the following (thislist is illustrative not exclusive):

-   -   1. Adding/using a 3-D visualization tool at the “pre-AFIS” stage        of an existing AFIS system. This assists the examiner to view        the fingerprint image in 3-D (e.g., the z-axis of the surface        representing grayscale magnitudes), plot AFIS minutiae markers        on the 3D surface, and then submit the minutiae markers to the        AFIS system for search.    -   2. As a variation of 1 above, the 3-D visualization tool also        assists the examiner to (a) add additional minutia after the        AFIS system has automatically generated minutiae and (b) adjust        the minutia placed by the AFIS system's automatic placement        feature.    -   3. For level III detail contained in a print, the 3-D        visualization of grayscale and other magnitudes herein assists        in the placement of minutiae.

The 3-D visualization tool also applies to Automated PalmprintIdentification Systems and other print-based identification systems, asexhibited for example in FIG. 6, which shows a 2D palm print 38 renderednormally (i.e., in 2D) and in 3D with the grayscale magnitudedemonstrated on an additional dimension, an embodiment of the systemsand methods discussed herein, for a more detailed 3D examination. Palmprints and fingerprints are both comprised of friction ridge skin andare searched in a similar fashion. Other desired prints, typicallyhuman, can also be imaged and used with the methods and systems herein,for example toe prints, foot prints, non-ridged skin prints, etc.

FIGS. 7–11 show screen shots of an embodiment of the AFIS 3D minutiaemarking software, and illustrate exemplary steps taken to mark ridgeendings and bifurcations (level II detail).

FIG. 7 is a screenshot of a fingerprint being plotted with an embodimentof the 3D minutiae marking tools discussed herein. As can be seen inFIG. 7 (and in some other figures), in some embodiments the placement ofminutiae markers on the 3D image can be simultaneously displayed on the2D image. If desired, as shown, the 2D image can be an inset next to orwithin the 3D image, or the images can be side-by-side, or otherwiselocated as desired.

Next, FIG. 8 depicts the same palm print image 38 as FIG. 6, goingthrough the minutia marking process. The first image represents a 2Dlatent palm print opened in minutia plotting software. When minutia areplaced in the 3D image, they automatically appear on the correspondingspot on the 2D image to the left.

FIG. 9 depicts the same palm print image 38 as FIG. 6, showing the imageundergoing rotation, skew, roll, etc., in any direction to facilitate anin-depth 3 dimensional examination of ridge structure 44.

FIG. 10 depicts the same palm print image 38 as FIG. 6 viewed throughenhancement filters such as contrast adjustments to provide a filteredimage 45. Such filters can be used at any desired time to furtherincrease visual clarity.

FIG. 11 depicts the same palm print image 38 as FIG. 6 wherein a usercan also place minutia markers 46 on the 3D surface that will bequantified and searched against existing 2D databases.

Thus, in some embodiments the methods and related software and othersystems herein comprise analyzing prints comprising: a) providing an atleast 2-dimensional image of a print; b) subjecting the image tomagnitude enhancement analysis such that at least one relativemeasurement across at least a substantial portion of the print can bedepicted in an additional dimension relative to the at least2-dimensions to provide a magnitude enhanced image such that additionallevels of that magnitude can be substantially more cognizable to a humaneye compared to the 2-dimensional image without the magnitudeenhancement analysis; c) displaying the enhanced image; and d) manuallyreviewing the magnitude enhanced image to place at least one minutiamarker on the print.

The placing can comprise identifying and placing at least one, two ormore minutiae markers not previously identified on the print, and cancomprise moving at least one, two or more minutiae markers previously,incorrectly placed on the print.

The methods can also comprise analyzing prints, comprising: a) providingan at least 2-dimensional image of a print comprising minutiae markersdetermined by an automated minutia marker algorithm to provide automatedminutiae markers; b) subjecting the image to magnitude enhancementanalysis such that at least one relative magnitude across at least asubstantial portion of the print can be depicted in an additionaldimension relative to the at least 2-dimensions to provide an magnitudeenhanced image such that additional levels of the chosen measure ormagnitude can be visible to a human eye compared to the 2-dimensionalimage without the magnitude enhancement analysis; c) displaying theenhanced image; and d) manually reviewing the magnitude enhanced imageto evaluate the correctness of the automated minutiae markers. Themethods can further comprise placing at least one minutia marker of theprint, the placing comprising at least one of removing incorrectautomated minutiae markers, moving incorrect automated minutiae markers,or adding further minutiae markers.

AFIS III+ Methods and Systems

When specific grayscale or other magnitudes in a fingerprint image areconnected, the resulting pathway conforms to specific level III edge andpore ridge features. Using multiple grayscale or other magnitudepathways enhances the recognition and extraction of more, and sometimessubstantially all, the level III features in an impression, therebyincreasing the likelihood of an AFIS hit.

By connecting pixels which possess the same grayscale or othermagnitude, a pathway or contour is formed. This pathway conforms tounique ridge shapes that are present along the edges, pores, and surfacemorphology of a friction ridge impression. The charted course and,generally, changes in the charted course of the pathway on the x-y axiscan be used to recognize and extract level III features.

When a different grayscale value (or other value for the measuredmagnitude) is chosen, the pathway or contour takes on a new course.Changes in the charted new course will not be the same as changes in anyother pathway course. In short, as the magnitudes used to chart thepathway change, the shape, location, prominence, and presence offeatures along that pathway also changes. If multiple magnitude pathways(multiple pathways within a given measurement indicator, such asgrayscale, and/or multiple pathways within or between differentmagnitude indicators such as grayscale and hue and saturation) are usedin an AFIS III+ environment, many, and possibly substantially all,features present in an impression may be recognized.

Level II AFIS (AFIS II) models mainly take into account major frictionridge path deviations in an impression. These include bifurcations 48and ridge endings 50, as demonstrated in FIGS. 12 a–c. FIG. 12 a (left)depicts an image of a known impression, FIG. 12 b (center) shows an AFISII system reading ridge paths and major ridge path deviations, then inFIG. 12 c (right) recognizing and extracting level II detail withdirectionality and relationship.

Level III AFIS (AFIS III) additionally takes into account features alongthe edge of a minor deviation such as a ridge positions 52, 54 and porepositions 56 along the center of a ridge, if available in the impressionof the print. FIG. 13 a (left) depicts an image of a known impression,FIG. 13 b (center) shows an AFIS II system reading ridge paths and majorridge path deviations, then in FIG. 13 c (right) recognizing andextracting level II detail with directionality and relationship.

AFIS III+ takes into account this detail and more, at multiple levels orslices of an impression, as shown in FIG. 14. These slices are definedby the grayscale values that are used to chart the course of contours orpathways within the slice. Thus, FIG. 14, AFIS III+, utilizes slicescomprising multiple image pathways which conform differently to levelIII features at different grayscale (or other) magnitudes.

As shown in FIGS. 15 a–c and FIGS. 16 a–c, these AFIS III+ pathways canbe isolated and examined individually to demonstrate the uniqueness ofthe friction ridge that created the impression. FIGS. 15 a–c depict anAFIS III+ analysis the same images as FIGS. 12 a–c at differentgrayscale magnitudes; FIGS. 16 a–c depict an AFIS III+ analysis the sameimages but at a different grayscale magnitude from the analysis in FIGS.15 a–c. By examining the course of the pathway and specifically, changesin the course of a pathway, unique features of that course can berecognized in the different slices (FIGS. 15 b and 16 b). By assigningvalue to changes in the course of a pathway, those unique features canbe extracted and used for searching (FIGS. 15 c and 16 c). Thus, inFIGS. 15 a and 16 a, differing individual pathways are seen. In FIGS. 15b and 16 b, features in the filled pathways (or slice) are recognized.In FIGS. 15 c and 16 c, only the marked features are shown(relationship, directionality, and prominence can be associated witheach feature).

Thus, if the pathway or level in the print is charted according to adifferent grayscale or other magnitude, the position, location,prominence, and presence of features along the pathway changes.

By charting pathways based on a comprehensive set of grayscale values inan image, very large amounts the detail present can be recognized,extracted, and used in the comparison and identification of afingerprint image. AFIS III+ results in a much more accurate andcomplete latent print feature-based profile, but that profile canrequire significantly more processing time and power to search. This isbecause the resulting feature set would contain many times more datathan traditional level 2 AFIS systems capture. Further, this increasedamount of data would be present on each slice of each image compared,and each slice would be compared with the multiple slices of knowndatabase images, as shown in FIG. 17.

Thus, in some embodiments only selected slices are used, or programs canbe provided to compress data or otherwise facilitate data storage,management, processing, analysis, etc.

Turning to a more general discussion of this aspect of the innovationsherein, one feature involves the use of multiple slices of a singleimage in an AFIS environment. Another aspect comprises the use ofpathways defined by grayscale or other magnitudes within an image of afriction ridge impression. Once the pathways in each slice aredetermined, any extraction or matching algorithm may be used to gatherand compare the data. Additional aspects comprise defining the featuresfor recognition.

There are typically four types of level III features involvingdirectional changes of grayscale magnitude pathways which can be presentand quantified in an AFIS III+ environment.

-   -   1) EC: point of maximum offset on a concave edge feature    -   2) EV: point of maximum offset on a convex edge feature    -   3) MC: center of mass of a concave morphological feature    -   4) MV: center of mass of a convex morphological feature

A morphological feature is a feature in which a contour line or pathwayforms a circuit around a level III feature. FIG. 18 shows examples of aconcave morphological feature that may include a sweat pore, adepression in the top of a ridge or other morphological feature wheredetail is a lighter grayscale shade than the surrounding dark pixels.Examples of a convex morphological feature include an incipient ridge ora bump on a friction ridge where detail shows up as a darker grayscaleshade than the surrounding light detail. In these Figures, the concavemorphological features are of a lighter grayscale value, and the convexmorphological features are of a darker grayscale value than surroundingdetail.

As shown FIG. 19, edge features are features along the pathwayproceeding down the edge of a friction ridge, represented by changes indirection of the pathway. Examples of a concave edge feature include aninlet of light (furrow) detail into a friction ridge or a sweat porethat is not quite closed in on one edge. Examples of a convex edgefeature include a bump on the side of a ridge or a section of a ridgethat protrudes into the furrow. In FIG. 19, the purple features areexamples of concave edge features, and the green features are examplesof convex edge features.

Recognition:

Through simple algebra or otherwise as desired, the location anddirection of each feature can be determined and plotted in relation tothe center of the pattern and other features on the x-y axis.

In the case of morphological features, center of mass and feature areacan be calculated, and directionality can be assigned based on pixelgrayscale magnitude relative to surrounding values. For example, convexfeatures would receive a “+” value and convex features would receive a“−” value. Noise can be reduced by analysis of the prominence of thefeature throughout multiple slices (subtle changes can be disregarded,or a threshold tolerance can be set). Noise can also be reduced byanalysis of the percentage change of grayscale pixel magnitude insurrounding pixels (sharp changes would represent artificial featuresthat are not friction ridge skin features).

For edge features, the point on a pathway that is furthest from theaverage path can be calculated, and directionality can be assigned basedon pixel grayscale magnitude relative to the value on either side of thepathway. Convex features would receive a “+” value and concave featureswould receive a “−” value. Noise can be reduced by analysis of thedeviation of the point from the average pathway (subtle changes can bedisregarded, or a threshold of tolerance can be set). Noise can also bereduced by analysis of the frequency of features along a pathway(frequent features would represent artificial features that are notfriction ridge skin features).

Turning to some general issues, the development of the innovationsherein have the potential to significantly increase the accuracy ofautomated fingerprint identification systems, and/or increase theidentification of more foreign and domestic criminals, therebycontributing to the advancement of law enforcement, criminal justicesystems and homeland security efforts.

Virtually any dimension, or weighted combination of dimensions in an atleast 2D digital image (e.g., a direct digital image, a scannedphotograph, a screen capture from a video or other moving image) can berepresented as at least a 3D surface map (i.e., the dimension orintensity of a pixel (or magnitude as determined by some othermathematical representation or correlation of a pixel, such as anaverage of a pixel's intensity and its surrounding pixel's intensities,or an average of just the surrounding pixels) can be represented as atleast one additional dimension; an x,y image can be used to generate anx,y,z surface where the z axis defines the magnitude chosen to generatethe z-axis). For example, the magnitude can be grayscale or a givencolor channel.

Other examples include conversion of the default color space for animage into the HLS (hue, lightness, saturation) color space and thenselecting the saturation or hue, or lightness dimensions as themagnitude. Converting to an RGB color space allows selection of colorchannels (red channel, green channel, blue channel, etc.). The selectioncan also be of single wavelengths or wavelength bands, or of a pluralityof wavelengths or wavelength bands, which wavelengths may or may not beadjacent to each other. For example, selecting and/or deselectingcertain wavelength bands can permit detection of fluorescence in animage, or detect the relative oxygen content of hemoglobin in an image.The magnitude can be determined using, e.g., linear or non-linearalgorithms, or other mathematical functions as desired.

Thus, the height of each pixel on the surface may, for example, becalculated from a combination of color space dimensions (channels) withsome weighting factor (e.g., 0.5*red+0.25*green+0.25*blue), or evencombinations of dimensions from different color spaces simultaneously(e.g., the multiplication of the pixel's intensity (from the HSI colorspace) with its luminance (from a YUV, YCbCr, Yxy, LAB, etc., colorspace)).

The pixel-by-pixel surface projections are in certain embodimentsconnected through image processing techniques to create a continuoussurface map. The image processing techniques used to connect theprojections and create a surface include mapping 2D pixels to gridpoints on a 3D mesh (e.g., triangular or rectilinear), setting thez-axis value of the grid point to the appropriate value (elevating basedon the selected metric, e.g., intensity, red channel, etc.), filling themesh with standard 3D shading techniques (gouraud, flat, etc.) and thenlighting the 3D scene with ambient and directional lighting. Thesetechniques can be implemented for such embodiments using modificationsin certain 3D surface creation/visualization software, discussed forexample in U.S. Pat. Nos. 6,445,820 and 6,654,490; U.S. patentapplication Ser. Nos. 20020114508; 20020176619; 20040096098;20040109608; and PCT patent publication No. WO 02/17232.

The present invention can display 3D topographic maps or other 3Ddisplays of color space dimensions in images that are 1 bit or higher.For example, variations in hue in a 12 bit image can be represented as a3D surface with 4,096 variations in surface height.

Other examples of magnitude and/or display option include, outside ofcolor space dimensions, the height of a gridpoint on the z axis can becalculated using any function of the 2D data set. A function to changeinformation from the 2D data set to a z height takes the form f(x, y,image)=z. All of the color space dimensions are of this form, but therecan be other values as well. For example, a function can be created inLumen software that maps z height based on (i) a lookup table to aHounsfield unit (f(pixelValue)=Hounsfield value), (ii) just on the 2Dcoordinates (e.g., f(x,y)=2x+y), (iii) any other field variable that maybe stored external to the image, or (iv) area operators in a 2D image,such as Gaussian blur values, or Sobel edge detector values.

In all cases, the external function or dataset is related in somemeaningful way to the image. The software herein can contain a functiong that maps a pixel in the 2D image to some other external variable (forexample, Hounsfield units) and that value is then used as the value forthe z height (with optional adjustment). The end result is a 3Dtopographic map of the Hounsfield units contained in the 2D image; the3D map would be projected on the 2D image itself.

Thus, the magnitude can be, for example, at least one or more ofgrayscale, hue, lightness, or saturation, or the magnitude can comprisea combination of magnitudes derived from at least one of grayscale, hue,lightness, or saturation, an average defined by an area operatorcentered on a pixel within the image. The magnitude can be determinedusing a linear or non-linear function.

The methods can also comprise performing one or more of AFIS II-type,AFIS III-type and/or AFIS III+-type analyses of the markers in theimage.

The at least 2-dimensional image of a print can be a 2-dimensional imageand the additional dimension relative to the 2-dimensions can be a thirddimension, to provide a 3D image having a 3-dimensional surface with thetwo dimensions of the 2-dimensional image represented on the x, y axesand the third dimension represented on the z-axis.

The magnitude analysis can differentiate sufficient levels of the valueto distinguish level III characteristics of the print. The magnitudeenhancement analysis can be a dynamic magnitude enhancement analysis,and can comprise rolling, tilting and/or panning the image. The dynamicanalysis can also comprise incorporating the dynamic analysis into acine loop, which indicates a video or other moving picture wherein aparticular roll, tilt or pan is reiterated back and forth; other optionsfor the cine loop include varying the aspect ratio of the surface from 0to some other number (positive or negative), varying the lightingparameters (e.g., the % mix of ambient and directional light), the angleof the directional lighting applied to the surface (“sweeping” thelights over the surface), etc. The image can be a digital image,photographic image, color image, or black and white image. The print canbe a fingerprint, palmprint, partial print, latent print.

The methods further can comprise, upon placing the minutia marker of theprint, simultaneously displaying the minutia marker on both a 2D imageand a 3D image of the print. The 2D image and the 3D image of the printcan be simultaneously displayed on a single display screen.

The innovations herein also comprise computer-implemented programmingthat performs the automated elements of the methods, and computerscomprising such computer-implemented programming. The computers cancomprise a distributed network of linked computers, such as a handheldwireless computer, and the methods can be implemented on the handheldwireless computer. The innovations also comprise an AFIS systemcomprising computers and/or computer-implemented programming thatperforms the methods herein.

In some embodiments, the image data modeling programs herein keep apersistent record, or command history, of every command performed on theoriginal data set. To return to a previous view of the surface object,the corresponding command is selected from the command history drop downlist, or via other mechanisms, such as striking the “undo/redo” commandkey several times, or otherwise as desired. When a magnitude enhancedimage is saved, the command history, and the current position in thehistory, is saved as part of the image file format or other inextricablylinked format. Such saving of the command history can be automatic ormanual, and can be mandatory or optional. For example, where theforensic history and/or chain of custody of the sample (and tests on thesample) are desirable, such as in the review of evidence forpresentation in court proceedings, the command history can be maintainedas a mandatory (i.e., can't be turned off by the user), automaticfeature that records every image manipulation for later review byopposing attorneys or experts, or other authorities. Similarly, incorporate settings where employee actions need to be tracked, thecommand history can be mandatory. In other settings, where verifiablehistory is not critical or desired, the command history function caneither be turned off or can be erased. It is a feature of desiredembodiments that the command history can never be “faked.”

In some embodiments, the saved command history will automaticallyappear, and can be applied in the same sequenced order, whenever thesaved data modeling file is opened. Thus, the command history is“persistent”, in that it remains part of each visualization file that issaved in the data modeling file after commands have been entered.

Once the command history is saved, the history by itself or with a copyof the image can be mailed to another user, who can then import it orotherwise access it, and use the saved command set on the sameunderlying image. This generates an identical visualization(s) withoutactually having to send the visualization(s) back and forth. This can beadvantageous, for example, where a crime lab wants an outside expert tolook at an image using the image modeling system, but ultimately wantsthe rendered visualization to be done in-house or does not have thecapacity to readily transmit or accept very large computer files (or aseries of such files). It also enhances the ability of the seconduser(s) to cross-check the methodology used by the original user. Thus,the image can be examined by one user to achieve a desired magnitudeanalysis, a pitch, roll and yaw, and/or other display settings to yielda desired enhanced image visualization, and then the command historyproviding such desired enhanced image visualization can be transmitted,for example by e-mailing, to the second user. They can then bring up thesame underlying image in the software installed on their computers orotherwise, plug in or copy the command history sent them, and create theexact same rendering. Additional renderings can also be then be made,and sent back to the original user or on to other users as desired. Insome embodiments, the first user is a central resource, such as theprovider of the image modeling software, that has particular expertisein the examination of the image or of the type of image. In otherembodiments, the users can be a plurality of different crime labs,medical labs, pathologists, or other users having a specific expertisearea but not specifically tied to the image modeling software. Otherusers are also possible as desired.

In another aspect, prints and other images discussed herein can beadvantageously displayed such that images with 9-bit or more magnitudesfor each pixel channel information can be displayed on 8-bit or lessdisplay systems. Tools that can be included in such methods andsoftware, etc., include surface/wireframe/contour/grid point mapping,contour interval controls, elevation proportions and scaling,pseudocolor/grayscale mapping, color/transparency mapping, surfaceorientation, surface projection perspectives, close-up and distantviews, comparison window tiling and synchronization, image registration,image cloning, color map contrast control by histogram equalize andlinear range mapping. The systems, etc., transform grayscale imageintensity, other magnitudes, to a 3D surface representation of themagnitude. The transformation results in a fundamental shift of HVSperception mechanisms, where tonal values are transformed into“elevation” shapes and forms corresponding to the chosen magnitude ofthe respective pixel. The elevation shapes and forms can be representedat any chosen contrast levels or hues, avoiding grayscale tonal displayand HVS perception issues. A variety of interactive tools and aids toquantitative perception can be used, such as zoom, tilt, pan, rotation,applied color values, isopleths, linear scales, spatial calibration, andmouse gesture measurement of image features.

The systems, etc., provide methods of displaying grayscale shades ofmore than 8 bits (more than 256 shades) and higher (16 bit, 65,536shades for example) on conventional display equipment, typically capableof a maximum of 8 bit grayscale discrimination. This is done in someembodiments by mapping the digitized magnitude image spatial informationon the X and Y axes of the image while plotting the grayscale value on aZ-axis or elevation dimension. The resulting three dimensional surfacecan assign any desired length and scale factor to the Z-axis, thusproviding display of grayscale information equal to or exceeding thecommon 256 grayscale limitation of printers, displays, and human visualperception systems.

Additionally, subsets of the full bit set of information (i.e.,completely uncompressed, or at least less compressed than the remainderof the magnitude information) can be displayed in a “magnificationwindow” such that only certain segments of the information are fullydisplayed while the remainder is compressed or even “left off” thedisplay screen. For example, the “window” can be a subset of the overallgrayscale range, 256 of 4096 for example. This “window” may be locatedto view grayscale values at midtone “level” (1920 to 2176), extremelydark “level” (0 to 255), or elsewhere along the 4096, 12 bit scale. Forthe extremely dark example, a 256 grayscale portion (window) ofextremely dark (level) grayscales from the 4096 or other high bit levelimage, would be adjusted to display those dark grayscales using midtonelevel grayscales readily visible to the HVS on common display equipment.The balance of 3840 grayscales (4096 minus 256) in the 12 bit imagewould not be visible on the display. By use of an optional 3 dimensionalsurface, the extremely dark shades are visible without adjustment(window and level), as well as the midtone and extremely light shades ofgray. All 4096 grayscale values will be available for HVS perception atone moment (or more, if desired) as 3D surface object.

Moreover, certain of the surface creation techniques, persistent commandhistory, display options, software, etc., discussed above and elsewhereherein themselves constitute innovations herein, including for purposesother than AFIS analyses. For example, such techniques, etc., can beuseful in medical, industrial, dental, forensic, quality assurance,personal identification, etc., situations.

All terms used herein, including those specifically discussed below inthis section, are used in accordance with their ordinary meanings unlessthe context or definition clearly indicates otherwise. Also unlessindicated otherwise, except within the claims, the use of “or” includes“and” and vice-versa. Non-limiting terms are not to be construed aslimiting unless expressly stated, or the context clearly indicates,otherwise (for example, “including,” “having,” and “comprising”typically indicate “including without limitation”). Singular forms,including in the claims, such as “a,” “an,” and “the” include the pluralreference unless expressly stated, or the context clearly indicates,otherwise.

A “computer” is a device that is capable of controlling a scanner,digital image analyzer, or processor or the like, or other elements ofthe apparatus and methods discussed herein. For example, the computercan control the AFIS analysis, the software discussed herein thatdetermines the grayscale or other magnitude and/or intensity slices,etc. Typically, a computer comprises a central processing unit (CPU) orother logic-implementation device, for example a stand alone computersuch as a desk top or laptop computer, a computer with peripherals, ahandheld, a local or internet network, etc. Computers are well known andselection of a desirable computer for a particular aspect or feature iswithin the scope of a skilled person in view of the present disclosure.

The scope of the present systems and methods, etc., includes both meansplus function and step plus function concepts. However, the terms setforth in this application are not to be interpreted in the claims asindicating a “means plus function” relationship unless the word “means”is specifically recited in a claim, and are to be interpreted in theclaims as indicating a “means plus function” relationship where the word“means” is specifically recited in a claim. Similarly, the terms setforth in this application are not to be interpreted in method or processclaims as indicating a “step plus function” relationship unless the word“step” is specifically recited in the claims, and are to be interpretedin the claims as indicating a “step plus function” relationship wherethe word “step” is specifically recited in a claim.

From the foregoing, it will be appreciated that, although specificembodiments have been discussed herein for purposes of illustration,various modifications may be made without deviating from the spirit andscope of the discussion and claims herein.

1. A method of analyzing prints comprising: a) providing an at least2-dimensional image of a print; b) subjecting the image to magnitudeenhancement analysis such that at least one relative magnitude across atleast a substantial portion of the print is depicted in an additionaldimension relative to the at least 2-dimensions to provide a magnitudeenhanced image such that additional levels of magnitudes aresubstantially more cognizable to a human eye compared to the2-dimensional image without the magnitude enhancement analysis; c)displaying the enhanced image; and, d) manually reviewing the magnitudeenhanced image to place at least one minutiae marker of the print. 2.The method of claim 1 wherein the placing comprises identifying andplacing at least two minutiae markers not previously identified on theprint.
 3. The method of claim 1 wherein the placing comprises moving atleast two minutiae markers previously, incorrectly placed on the print.4. A method of analyzing prints comprising: a) providing an at least2-dimensional image of a print comprising minutiae markers determined byan automated minutia marker algorithm to provide automated minutiaemarkers; b) subjecting the image to magnitude enhancement analysis suchthat at least one relative magnitude across at least a substantialportion of the print is depicted in an additional dimension relative tothe at least 2-dimensions to provide a magnitude enhanced image suchthat additional levels of magnitude are visible to a human eye comparedto the 2-dimensional image without the magnitude enhancement analysis;c) displaying the enhanced image; and, d) manually reviewing themagnitude enhanced image to evaluate the correctness of the automatedminutiae markers.
 5. The method of claim 4 wherein the method furthercomprises determining at least one minutia marker of the print, thedetermining comprising at least one of removing incorrect automatedminutiae markers, moving incorrect automated minutiae markers, or addingfurther minutiae markers.
 6. The method of claim 1 or 5 wherein themagnitude is grayscale.
 7. The method of claim 1 or 5 wherein themagnitude comprises at least one of hue, lightness, or saturation. 8.The method of claim 1 or 5 wherein the magnitude comprises a combinationof values derived from at least one of grayscale, hue, lightness, orsaturation.
 9. The method of claim 1 or 5 wherein the magnitudecomprises an average intensity defined by an area operator centered on apixel within the image.
 10. The method of claim 1 or 5 wherein themagnitude is determined using a linear function.
 11. The method of claim1 or 5 wherein the magnitude is determined using a non-linear function.12. The method of claim 1 or 5 wherein the method further comprisesperforming AFIS II-type analysis of the markers in the image.
 13. Themethod of claim 1 or 5 wherein the method further comprises performingAFIS III-type analysis of the markers in the image.
 14. The method ofclaim 1 or 5 wherein the image is a digital image.
 15. The method ofclaim 1 or 5 wherein the print is a fingerprint.
 16. The method of claim1 or 5 wherein the print is a palmprint.
 17. The method of claim 1 or 5wherein the print is a partial print.
 18. The method of claim 1 or 5wherein the print is a latent print.
 19. The method of claim 1 or 5wherein the method further comprises, upon placing the minutia marker ofthe print, simultaneously displaying the minutia marker on both a 2Dimage and a 3D image of the print.
 20. The method of claim 19 whereinthe 2D image and the 3D image of the print are simultaneously displayedon a single display screen.
 21. Computer-implemented programming thatperforms the automated elements of the method of claim 1 or
 5. 22. Acomputer comprising computer-implemented programming that performs theautomated elements of the method of claim 1 or
 5. 23. The computer ofclaim 22 wherein the computer comprises a distributed network of linkedcomputers.
 24. An AFIS system comprising computer-implementedprogramming that performs the automated elements of the method of claim1 or
 5. 25. An AFIS system comprising a computer according to claim 22.26. A method of analyzing prints comprising: a) providing an at least2-dimensional image of a print; b) subjecting the image to magnitudeenhancement analysis such that at least one relative magnitude across atleast a substantial portion of the print is depicted in an additionaldimension relative to the at least 2-dimensions to provide a magnitudeenhanced image such that additional levels of magnitude are visible to ahuman eye compared to the 2-dimensional image without the magnitudeenhancement analysis; c) dividing the magnitude enhanced image into aplurality of intensity levels; d) individually selecting at least oneisolated intensity level; e) determining at least one AFIS marker fromthe isolated intensity level.